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ABSTRACT 


This paper compares the students enrolled to a three-year undergraduate, bachelor degree on Security of Computer 
Systems and Networks — offered in traditional, classroom fashion as well as online at the University of Milan (Italy) — in 
terms of results in exams of the various courses as well as in the final dissertation for obtaining the degree. This gives 
useful information about the effectiveness of e-learning vs. traditional learning in presence, and the possibility of using 
e-learning as a vehicle for extending the enrollment to already existing traditional university degrees. 
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1. INTRODUCTION 


As discussed e.g. in Damiani (2005), Frati (2010), Milani (2014), Scarabottolo (2016-1) and Scarabottolo 
(2016-2), the University of Milan (Italy) offers an undergraduate, three-year degree on Security of Computer 
Systems and Networks in a university campus located in Crema, a small town 40 Kilometers east of Milan. 
Such a degree (from here on denoted as SSRI from its Italian name: “Sicurezza dei Sistemi e delle Reti 
Informatiche”) has been activated in academic year 2003/04 as a traditional, classroom based university 
degree, but starting from academic year 2004/05 it is also offered online. Being SSRI an ICT-centered 
degree, we expected students more prone to the use of e-learning than students of other less technological 
degrees. 

The online version of SSRI required a significant design process, involving a deep revision of all teaching 
materials already prepared for classroom lectures by teachers as well as of the role of the staff supporting the 
student learning activities. Such a process has been coordinated by CTU (the e-Learning Centre of the 
University of Milan) initially supported by consultants from Isvor Knowledge System (an Italian company 
specialized in the production of e-learning courses). Main characteristics of the resulting degree structure are 
the following: 

¢ lectures are supplied in form of videos mainly constituted by slide sequences and/or desktop capturing 
synchronized with teacher’s voice, or blackboard-like effects recording teacher’s voice and handwriting 
(almost no video recording of the teacher her/himself); 

¢ each topic of the various courses is covered by a video-lecture whose duration is around one fourth of 
the time spent by the teacher to present the same topic in classroom; in other words, the video-lecture is 
optimized for transferring the learning content in a short time, to maximize the attention of the online 
student, who has the opportunity to review the lecture several times; 

* to furtherly facilitate online students learning, the academic year is divided into three four-month 
periods, each accommodating learning activities corresponding to around 20 ECTS credits; on the 
contrary, classroom students follow the regular semester-based organization, with around 30 ECTS 
credits per semester; 

¢ students are supported by one Expert tutor for each single course of the degree and for each group of 
40/50 students; 

¢ in addition, the degree is supported by a Process tutor, who acts as e-moderator, process facilitator, 
adviser/counsellor for the whole community of online students; 
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¢ online students must come to the Crema campus for intermediate and final exams (by law, exams 
cannot be undertaken online) scheduled in reserved sessions on Friday afternoon and Saturday to 
facilitate participation of students already employed (the majority of online students); 

¢ if online students do not succeed in passing all exams in the reserved sessions, they must present 

themselves in regular exam sessions, together with classroom students; 

¢ performance of students in each single exam is given by a number ranging from 18 (fairly passed) to 30 

(excellent) eventually “cum laude” (outstanding); 

* when a student graduates (i.e., obtains the “laurea”) her/his overall performance is given by a number 

ranging from 66 (fair) to 110 (excellent) eventually “cum laude” (outstanding). 

To evaluate the effectiveness of the learning scheme adopted for SSRI online, this paper compares the 
results of the online students vs. the classroom students over eight years, from 2010 to 2017. Comparison is 
made in terms of grades obtained in the various exams, time passed between the end of each course and the 
corresponding exam, sequence of passed exams vs. sequence of courses in the study plan, grade obtained in 
the final degree. 


2. STRUCTURE OF THE DEGREE 


The structure of SSRI obviously complies with the general rules given by the Italian Ministry of Education 
and University for undergraduate degrees: the degree has a three-year duration and requires students to earn 
180 ECTS credits (around 60 credits per year) to graduate. The general approach to design these 
undergraduate degrees foresees basic and introductory courses during the first year, core courses of the 
discipline during the second year, specialization courses and elective courses plus preparation of the final 
dissertation during the third year. This general approach results for SSRI in the organization reported in 
Table 1: 17 mandatory courses (7 in 1“ and 2" year and 3 in 3™ year) plus 2 elective courses in 3™ year. 


Table 1. Organization of courses in the SSRI degree 


Year Course No of ECTS credits 

Mathematical Analysis 12 

‘AlgebraandGeometry 6 

“Probability and Statistics SC 6 

FIRST © Computer Architecture == SC ase 

‘Computer Programming ii ss—<—~dSt 

“Security in Web and Mobile Systems’ | Gece, Pea, 

‘Tlaw © 4 ns 
Algorithms and Data Structures 12 

“Databases sts—‘“‘i‘s™é™és*zdSSCSS! 

“ComputerNetworks sid 

SECOND | OperatingSystemsI 6 

“Operating Systems Sn 

“Cryptography 6 

' Technologies for Security and Privacy: 6 
Systems and Networks Security 12 

“Secure Software Design 6 

rep TT inet Management 

elective courses 6+6 
‘ personal skills, foreign language, etc. | 6 
‘final thesis preparation = 7 acta 


' This course has been introduced pretty recently in the curriculum of SSRI, and still there are not enough data to consider it in the 
following sections and figures. 
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As it can be seen: 

* the first year is mostly dedicated to mathematics and basic courses in computer architecture and 
computer programming; 

¢ the second year concentrates on core courses for every ICT professional, plus some insights into topics 
typical of computer security and privacy; 

¢ the third year deepens student knowledge in systems and networks security and management of IT 
incidents. Twelve ECTS credits are reserved to elective courses, while all other courses listed in Table 
1 are mandatory. 


3. GRADES OBTAINED IN EXAMS 


A first, significant comparison of the two categories of students is given in Figure 1, showing the average 
grades obtained in the exams of the courses listed in Table 1. 


Average grades obtained in exams (18-30) 
18 20 22 24 26 28 30 
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(Incident Managenent EE 


CN ———————E_ 


classroom students Oonline students 


Figure 1. Average grades obtained in SSRI exams 


Perhaps surprisingly, online students outperform classroom ones in the exam results for all courses of the 
degree, in some cases (e.g., Operating Systems I) with a difference of more than two points. Considering that 
the range of grades for passed exams is 12 points (from 18 to 30) this means that online students have an 
average performance up to 20% better than classroom students. 

Such a result is definitely a proof of the effectiveness of the didactical organization of SSRI online: the 
learning process based on the video-lectures, supported by the team of tutors and planned to concentrate on 
few courses in each four-month period allows students to come to exams well prepared and to pass them with 
better results. 

However, this does not imply that the learning process for classroom students (traditional lectures, no 
tutors, more courses in parallel due to the semester-based planning of courses) is less effective: results in 
exams are also strictly dependent on student motivation, i.e., commitment to improve personal knowledge 
and skills. To this regard, it may be argued that online students are far more committed than classroom ones 
mainly for two reasons. 
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1. As shown in Figure 2, the age of classroom students enrolling to SSRI online is completely different 
from that of classroom students. Classroom students are in large majority “conventional” young people, 
entering the university just after completion of the high schools. On the contrary, online students are 
definitely older people, generally employed, coming back to university to improve their professional 
profile. For them, study implies to sacrifice evenings and weekends with families and children, thus it is 
reasonable to expect far more motivation to succeed. 

2. Online students are requested to pay — besides the same tuition of classroom students, ranging from few 
hundreds to three thousands euros on the basis of the incomes of the student family — an additional fee 
of 1,500 euros for online services (tutorship, weekend exams, didactic material updating, etc.). Again, 
something pushing toward commitment... 


Percentage of students by age when enrolling to SSRI 
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Figure 2. Age of students enrolling to SSRI 


4. TIME TO PASS EXAMS 


Another comparison giving significant information about the two categories of students is the analysis of the 
average time required to prepare and to pass exams at the end of the lectures period, reported in Figure 3. 

A first, immediate consideration coming from data in Figure 3 is the very long average time (no too far 
from one year) that both categories of SSRI students require to pass exams. Such a bad situation is 
unfortunately not that surprising: the job market is definitely eager of IT professionals, and a significant 
percentage of IT students starts working (usually part time) before graduating, thus lengthening the study 
period. 

A second consideration seems to contradict what already said about commitment of online students: in 
fact, online students show longer preparation times for almost all exams with respect to classroom students. 
This contradiction disappears by limiting the analysis to the subset of students already graduated — i.e., 
students who passed all exams and obtained the final degree (“laurea’”’) — as shown in Figure 4. Besides a 
general reduction in the time required for passing the exams, it becomes evident that online students are 
generally faster than classroom ones, as expected given the previous considerations about their commitment. 

The fact that data in Figures 3 and 4 show completely different situations can be explained by considering 
the number of students whose data are used in these analyses, summarized in Table 2: 
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* part of the students considered in these analyses registered to SSRI in the most recent years of the 
2010-2017 period, thus they were still far from the end of the bachelor; this partially explains the low 
percentages of graduated students; 

* the significant length of the average time to obtain the final degree (around one year more than planned 
for a three year bachelor degree) — mainly due to the above considerations about part time work 
activities during studies — furtherly justifies the low percentages of graduated students; 

* the very low percentage of online students who obtained their final degree by the end of 2017 (less than 
one half of the percentage of classroom students) is mainly due to the far higher number of online 
students leaving the degree without terminating it, having underestimated the effort required to come 
back to university at an older age. 


Average time required for passing exams (no. of days) 
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Figure 3. Average time to prepare and pass SSRI exams after end of lectures 
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Figure 4. Average time to prepare and pass SSRI exams after end of lectures by graduated students 
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Table 2. Students used to perform the analyses reported in this paper 


| Classroom students | Online students 


Number of students considered in the analyses 315 341 
“Number of students graduated at theendof 2017. 7S Bs bor, 
“Percentage of students graduated at the end of 2017, 23.81% | *(10,85% 
“Timetograduate(no.ofdays) | I4il 1 317 


5. WHEN STUDENTS TAKE EXAMS 


Data in Figures 3 and 4 also show significant differences in the times required to pass the exams of the 
various courses, regardless to the number of ECTS credits (i.e., estimated study time) associated. For sure, 
not all topics are equally easy/difficult to learn, and (even more...) the effort to convince the teacher to give a 
positive grade is definitely not the same in all cases! 

However, another reason for those differences in times could be the decision of a student to postpone the 
exam of a course already terminated to concentrate on the lectures of other courses. In other words, it is 
possible for students not to fully adhere to the degree planning given in Table 1, delaying some exams 
considered harder, or less interesting, etc. Figure 5 presents three different approaches to the mandatory 
exams by students. 
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Figure 5. Position of some mandatory exams in the 17-exam plan of SSRI 
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A first possible behavior of students — represented in Figure 5 by IT Law — is the ideal situation: for most 
of the students, this exam is one of the first exams of their career, which requires passing 17 mandatory 
exams before discussing the final dissertation. 

A second possible behavior of students — represented by Probability and Statistics — is on the contrary the 
worst situation: even if the course is planned for the first year of the degree, it is taken by a significant 
percentage of students in the second and even in the third year. There are several possible reasons for that: 
difficulty of the topic (but not for this particular course, looking at the average grades in Figure 1) scarce 
interest of students for the topic, feeling that the topic is not functional to the comprehension of the following 
courses, etc. In any case, this requires reconsidering the study plan of the degree to limit this delaying. 

A third possible behavior of students — represented by Computer Programming — is a course showing 
quite a significant difference in the behavior of online vs classroom students. Here, most of the online 
students correctly pass the exam during the first year, while a significant percentage of classroom students 
delays the exam, even if the specific topic is definitely preparatory to second year exams. This is most likely 
due to the different organization of the calendar: online students follow courses in four-month periods, and 
can take intermediate tests, while classroom students follow courses in semesters without intermediate tests. 
Again, a redesign of the study plan and student evaluation procedure would limit the problem. 


6. QUICK MEANS GOOD? 


Last, it is possible to investigate about a possible correlation between time required to complete the degree 
and grade obtained after the final dissertation. Data relative to the students already graduated in SSRI allow 
to plot the graphs of Figure 6, and to draw the trend lines shown there. 


Time to graduate vs. Grade obtained after final dissertation 


Time to graduate (years) 
- uw 


w 


75 80 85 90 95 100 105 110 
Grade obtained after final dissertation (66-110) 


@ Classroom students © Online students Trend line (classroom students) = = = Trend line (online students) 


Figure 6. Correlation between time to graduate and final grade obtained 


The slope of the trend lines in Figure 6 is — as easily predictable — negative: “faster” students obtain final 
better final grades. However — contrary to what presumable — it is evident that the correlation between the 
two sets of values is very fair, especially for online students, whose dots in Figure 6 are less concentrated 
around the trend line than the ones of classroom students. In fact, the Pearson’s correlation coefficients p 
computed for the two categories of students are the following: 

P classroom — —0,4471 P online = —0,2256 
meaning as expected an inverse correlation, but moderate (—0,7 < p < —0,3) for classroom students and 
definitely fair (—0,3 < p < 0) for online students. 
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7. CONCLUSIONS 


This paper presented a comparison of performance in studies for two different categories of students, enrolled 
to the same undergraduate university degree but in two different versions: traditional learning (lectures in 
classroom) and e-learning (video-lectures available online, and tutorship using forums and emails). 

Such a comparison took into account students enrolled from 2010 to 2017, and considered results in 
exams (i.e., grades obtained at the end of each course), time to pass exams after the end of the lectures 
period, sequence of passed exams vs planned curriculum, time to reach the final degree and grade obtained. 

Main conclusions are the following. 

¢ Online students demonstrate a stronger commitment to study and pass exams than classroom students. 

In fact: 
— exam grades obtained by online students are higher in almost all courses; 
— time to pass exams for online students who are able to keep the correct study pace is almost always 
shorter. 

« A consequence of the above is that the online approach adopted for SSRI proved to be very effective, 

since it allows committed online students not to be penalized in their learning process. 

¢ The counterpart is that online students not sufficiently committed, or too busy in their professional life, 

tend to delay excessively exams and final dissertation. 

¢ Not all exams are passed when planned by the curriculum of the degree: especially classroom students 

tend to postpone some exams even if related to basic topics. 

¢ There is no significant correlation between time to graduate and grade obtained after the final 

dissertation: several “slow” students — both classroom and online — obtain excellent results. As already 
said, most IT students find part time job during their studies, thus delaying final dissertation regardless 
to their own skills. 
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